Recognition of numerical characters in digital images

ABSTRACT

Recognition of numerical characters is disclosed, including: extracting a subimage from a received image comprising information pertaining to a plurality of numerical characters, wherein the extracted subimage is associated with one of the plurality of numerical characters; and performing recognition based at least in part on a set of topological information associated with the subimage, including: processing the subimage to obtain the set of topological information associated with the subimage; comparing the set of topological information associated with the subimage with a preset set of stored topological information; determining that in the event that the set of topological information associated with the subimage matches the preset set of stored topological information, the subimage is associated with a recognized numerical character associated with the preset set of stored topological information.

CROSS REFERENCE TO OTHER APPLICATIONS

This application is a continuation of co-pending U.S. patent applicationSer. No. 13/199,332, entitled RECOGNITION OF NUMERICAL CHARACTERS INDIGITAL IMAGES filed Aug. 25, 2011 which is incorporated herein byreference for all purposes, which claims priority to People's Republicof China Patent Application No. 201010270456.1 entitled A DIGITAL IMAGERECOGNITION METHOD, DEVICE, AND A CRAWLER SERVER filed Aug. 30, 2010which is incorporated herein by reference for all purposes.

FIELD OF THE INVENTION

This present disclosure involves the field of image processing. Inparticular, it pertains to a technique of digital image recognition.

BACKGROUND OF THE INVENTION

Digital images are desired to be recognized in various contexts. Forexample, recognition of digital images may need to be employed by webcrawlers. Web crawlers (herein sometimes referred to as just “crawlers”)collect data to facilitate search engines in performing their searches.A web crawler typically downloads web pages (using a computer) and thenextracts and/or recognizes data that is included or hidden within theweb pages. The web crawler can arrange the extracted/recognized datainto a format that can be accepted/accessed by the search engine to usewhen building an index. As such, the web crawler's task of extractingand/or recognizing data from a web page is important.

For example, in the context of a vertical search engine (i.e., a searchengine that focuses on a particular segment of content, as opposed toall types of content) for businesses, the associated web crawler canextract and/or recognize such information as: “store name,” “address,”“phone number,” “map coordinates,” and “reviews.” For example, phonenumbers (e.g., including company, business, or personal phone numbers)are very important pieces of information in the vertical search becauseinclusion of accurate phone numbers can greatly contribute to theinformation quality and effectiveness of a particular business. In thisexample, even if the extraction and/or recognition of information iscomplete for the business other than the phone information, the datacollected for the business is still deficient because of the lack ofaccurate phone number information. Another example of a series ofnumbers that is desired to be recognized can be an ID number that ispresent in a scanned image (e.g., a scanned image of a ID card).

Continuing with the phone information example, a phone number isgenerally a string of numerical characters. Sometimes, at a web page, aphone number is presented in the form of an image. This is so that thephone number does not interfere with a user's reading. Also, since thephone number itself is relatively short and takes up only a very smallarea on a web page, when the phone number is transmitted as a picture,it does not unduly increase network overhead. If information a webcrawler needs to obtain, such as phone numbers, is stored as a digitalimage, then it is useful for the web crawler to include an opticalcharacter recognition (OCR) functionality to recognize such (e.g.,numerical) information. OCR is a form of mechanical and/or electronictranslation of scanned images of text into machine-intelligible text.

OCR is one form of computer pattern recognition and the recognition ofonly digits (any numerical characters between “0” to “9”) is aparticular branch of OCR. Typically, the available technologies employ adigit recognition technique that is a differentiation technique. Such adifferentiation technique usually implements the following steps:receive image models for individual digits “0”-“9” and separatelydifferentiate an image to be recognized from the individual modelimages, find the number of different pixels between the image to berecognized and the model image, and the digit that corresponds to themodel image with the smallest number of pixels different from the imageto be recognized is determined to be the numerical character in theimage to be recognized. This method has a good level of accuracy ofrecognition for images that are not geometrically distorted. However, ifthe image noise is severe (e.g., the image is noisy even after applyinga noise removal technique), the quality of the digit recognition thatuses the differentiation technique may be degraded. Also, thedifferentiation technique may not be as helpful in processing images ofdigits with geometrical deformations (e.g., images that are warped orare zoomed in).

Another typical method of image recognition includes the use of neuralnetworks. An image recognition technique that includes the use of neuralnetworks typically includes the following general steps: featureextraction is performed on the image to be recognized and/or also adescription is made of the features to be recognized, somehuman-recognized samples are selected to serve as objects for machinelearning, and the machine learning technique can output the image mode(e.g., recognition rules or patterns). By applying this machine learnedmode with regard to the images to be recognized, the numericalcharacters can be obtained in the images to be recognized. Utilizing theneural network technique in digital image recognition also involvescertain issues. However, the recognition rate of individual numericalcharacters in the neural network technique is higher than in thedifferentiation technique (e.g., it can be as high as 96%˜98% forindividual character recognition using the neural network technique).For example, in practice, China-based landline phone numbers usuallyconsist of at least of 8 digits, and China-based cell phone numbers aregenerally made up of even more digits. Assuming there is a 96%˜98%accuracy for individual characters, the accuracy of image recognitionfor an 8-digit phone number (such as for a China-based stationary phone)would be approximately 72.1˜85.1%, the accuracy of image recognition foran 11-digit phone number (such as for a China-based cell phone), theaccuracy of image recognition would be approximately 63.8˜80.1%, and theaccuracy of image recognition for a 12-digit phone number (such asincluding the area code with the phone number of a stationary phone) theaccuracy would be approximately 61.3˜78.5%. In practice, when imagerecognition accuracy is not high, the recognition results provided by acrawler are very likely to be of poor quality (e.g., includes manyincorrectly recognized characters). To achieve better accuracy of imagerecognition, the neural network model(s) of the recognition program canbe repeatedly tweaked and improved. However, repeatedly tweaking theimage recognition model can be inefficient.

BRIEF DESCRIPTION OF THE DRAWINGS

Various embodiments of the invention are disclosed in the followingdetailed description and the accompanying drawings.

FIG. 1 shows an example image of the numerical character of “3.”

FIG. 2 is a diagram showing an embodiment of a system for numericalcharacter recognition.

FIG. 3 is a flow diagram showing an embodiment of a process of numericalcharacter recognition.

FIG. 4 shows an example of a subimage before and after it has beenrefined.

FIG. 5 is a flow diagram showing an embodiment of a process of numericalcharacter recognition.

FIG. 6A shows an example of the vertices of a subimage with thedepiction of numerical character “4.”

FIG. 6B shows three examples of a center filled in pixel of the subimageand its respective eight surrounding pixels.

FIG. 7 shows an example image of a numerical character “4” with acentral point at the center of a closed ring.

FIG. 8 shows an example of one form in which preset sets of topologicalinformation and their corresponding numerical characters can be storedin the database.

FIG. 9 shows a diagram of an embodiment of a system for processingnumerical character recognition.

FIG. 10 is a diagram showing an embodiment of a system for processingnumerical character recognition.

DETAILED DESCRIPTION

The invention can be implemented in numerous ways, including as aprocess; an apparatus; a system; a composition of matter; a computerprogram product embodied on a computer readable storage medium; and/or aprocessor, such as a processor configured to execute instructions storedon and/or provided by a memory coupled to the processor. In thisspecification, these implementations, or any other form that theinvention may take, may be referred to as techniques. In general, theorder of the steps of disclosed processes may be altered within thescope of the invention. Unless stated otherwise, a component such as aprocessor or a memory described as being configured to perform a taskmay be implemented as a general component that is temporarily configuredto perform the task at a given time or a specific component that ismanufactured to perform the task. As used herein, the term ‘processor’refers to one or more devices, circuits, and/or processing coresconfigured to process data, such as computer program instructions.

A detailed description of one or more embodiments of the invention isprovided below along with accompanying figures that illustrate theprinciples of the invention. The invention is described in connectionwith such embodiments, but the invention is not limited to anyembodiment. The scope of the invention is limited only by the claims andthe invention encompasses numerous alternatives, modifications andequivalents. Numerous specific details are set forth in the followingdescription in order to provide a thorough understanding of theinvention. These details are provided for the purpose of example and theinvention may be practiced according to the claims without some or allof these specific details. For the purpose of clarity, technicalmaterial that is known in the technical fields related to the inventionhas not been described in detail so that the invention is notunnecessarily obscured.

The techniques of the present disclosure can be used in a multitude ofuniversal or specialized computer device environments or configurationsuch as in, for example, personal computers, server computers, handhelddevices or portable devices, tablet type devices, multiprocessingdevices, a distributed environment including any one or more of theabove-mentioned devices or devices, etc.

In some embodiments, the present disclosure can be implemented ascomputer executable instructions executed by a computer or a programmodule, for example. Generally, a program module includes the executionof a specific task or the implementation of a specific abstract datatype routine, program, object, group, data structure, etc. This presentdisclosure can also be realized in a distributed computer environment,and in such a distributed computer environments, tasks can be executedby remote processors connected via communication networks. In adistributed computer environment, program modules can be placed in localand remote computer storage media, including storage devices.

Techniques of image recognition are disclosed. In various embodiments,the image recognition is applied in the context of recognizing numericalcharacters or symbols. The following example introduces an underlyingconcept of the present application: human visual recognition ofnumerical characters can recognize a numerical character, such as “3,”regardless of the formatting of the character. For example, regardlessif “3” appears as 3 (in bold), 3 (written in a larger font) or 3(italicized), a human eye can determine that what it sees is still a“3.” Other slight variations in the character would not necessarilyrender the character unrecognizable.

FIG. 1 shows an example image of the numerical character of “3.” In theexample, the 3 black dots represent the vertices of the numericalcharacter “3” that are shared by all variations of the printed “3” andlines between the dots represent the edges between the vertices.Regardless of some variations of the edges between the vertices that maycause the “3” to becomes warped or deformed, the process of human visualrecognition can still recognize the numerical character so long as thevertices of the numerical character are present, correctly positionedrelative to each other, and correctly connected by the edges. Hence, thevertices and the edges play an important role in the process ofnumerical character recognition. It is assumed that so long as thevertices and edges between the vertices are available for a numericalcharacter, then that numerical character can be recognized.

FIG. 2 is a diagram showing an embodiment of a system for numericalcharacter recognition. System 200 includes servers 202, 204, 206,network 208, web crawler 210, and preset sets of topological informationdatabase 212. Network 208 can include high-speed data and/ortelecommunications networks.

Servers 202, 204, and 206 are examples of web servers that host websitesthat include web pages that web crawler 210 is configured to crawl.Servers 202, 204, and 206 can be operated by third parties.

Web crawler 210 is configured to crawl web pages (e.g., hosted byservers such as 202, 204, and 206). In some embodiments, web crawler 210can be implemented using one or both of hardware and software. In someembodiments, web crawler 210 is associated with a search engine and/oris a component of a search engine. In various embodiments, web crawler210 is configured to download content from the web pages for laterprocessing by an associated search engine that will index the downloadedweb pages to provide better searches. In some embodiments, web crawler210 is also configured to extract specific information from thedownloaded web pages. For example, web crawler 210 can be configured toextract certain information (e.g., numerical characters) from certaindownloaded images. Specifically, certain downloaded images depictingnumerical characters (e.g., phone numbers) would be helpful for webcrawler 210 to recognize (e.g., for the search engine's indexing orother purposes).

In some embodiments, web crawler 210 is configured to performrecognition of numerical characters on images (e.g., images downloadedfrom web pages). Web crawler 210 is configured to extract one or moresubimages from an image (e.g., an image obtained from a crawled webpage). The image can include one or more depictions of numericalcharacters and each extracted subimage includes a depiction of onenumerical character. In some embodiments, web crawler 210 is configuredto perform one or more forms of preprocessing (e.g., conversion tograyscale, noise removal) and/or refinement (e.g., image degradation) onthe subimages. Web crawler 210 is configured to extract topologicalinformation from each of the one or more refined subimages. As usedherein, topological information refers to properties of a shape that arepreserved despite deformities to the shape. Topological information withrespect to a depiction of a numerical character describes properties ofthe character that are preserved despite any deformation that thecharacter may go through. Topological information can include, forexample, the vertices and edges, the degree for each individual vertex,if the vertices and edges form one or more closed rings, and whether twovertices are connected by an edge in the depiction in each subimage. Webcrawler 210 is configured to compare the extracted topologicalinformation of a subimage against preset sets of topological informationstored at database 212. Each preset set of topological informationstored at database 212 is associated with a numerical character (e.g.,“0” to “9”). If a match is found between the extracted topologicalinformation of a subimage and a preset set of topological information,then the depiction in the subimage is determined to be recognized as thenumerical character corresponding to the matching preset set oftopological information. In some embodiments, if no match is foundbetween the extracted topological information of a subimage and anypreset set of topological information, then the extracted topologicalinformation is stored (e.g., to be analyzed later).

FIG. 3 is a flow diagram showing an embodiment of a process of numericalcharacter recognition. In various embodiments, process 300 can beapplied to recognizing individual numerical characters (e.g., “0” to“9”). In some embodiments, process 300 can be implemented at system 200.In some embodiments, process 300 can be implemented at system 900.

At 302, a subimage is extracted from a received image, wherein theextracted image is associated with one numerical character.

In various embodiments, the received image can be retrieved from a webpage crawled by a web crawler. In some embodiments, the received imagehas already been screened to include one or more numerical charactersand those received images that do not include one or more numericalcharacters are not further processed for numerical characterrecognition. For example, an image can be screened to determine if itincludes a numerical character by analyzing the structure of a web pageto extract only images at locations of the web page where it isstatistically likely to include images depicting numerical characters.In a received image with one or more numerical characters, eachnumerical character is depicted as an image to be recognized. Forexample, an image with multiple numerical characters can be an image ofa phone number (e.g., a series of individual numerical characters). Asused herein, a subimage refers to an image that is extracted fromanother image and includes at least a portion of the original image fromwhich it was extracted. The received image can be extracted into as manysubimages as there are depictions of numerical characters (i.e., eachdepiction of a numerical character can be extracted into one subimage).For example, if there were 10 numerical characters depicted in thereceived image, then 10 corresponding subimages can be extracted fromthe image. In some embodiments, a depiction of a numerical character isdetected if there is enough white space/lack of pixel content on bothsides of such a depiction. In some embodiments, the received image candepict characters/symbols other than numerical characters (e.g.,parentheses “(” or “)”) and each of these characters/symbols can also beextracted into a subimage.

In some embodiments, prior to extracting subimages from the receivedimage, preprocessing is performed on the received image. For example,color differences included in the received image can be removed (e.g.,using filters) such that the image includes only the colors of black,white, and/or gray (i.e., the image becomes a monochrome or grayscaleimage). Also, for example, at least some of the noise in the image canbe removed. Noise can refer to image content other than the imagecontent that is helpful to facilitating the recognition of numericalcharacters, such as redundant pixel information. The purpose ofperforming preprocessing on the received image is to remove noise andother factors that could potentially degrade the quality of numericalcharacter recognition.

At 304, recognition is performed on the subimage based at least in parton topological information associated with the subimage, includingprocessing the subimage to obtain the set of topological informationassociated with the subimage, comparing the set of topologicalinformation associated with the subimage with a preset set of storedtopological information, and determining that in the event that the setof topological information associated with the subimage matches thepreset set of stored topological information, the subimage is associatedwith a recognized numerical character associated with the preset set ofstored topological information. If there were more than one subimageextracted at 302, then the recognition process at 304 can be performedfor each subimage either in parallel or successively. In someembodiments, 304 is performed by the web crawler.

In some embodiments, recognition of a subimage includes refining thesubimage. As mentioned above, in some embodiments, preprocessing such asnoise removal and grayscale conversion have already been performed onthe subimage. The preprocessed subimage can be further refined using aconventional technique of image “degradation” from the field ofmathematical morphology. The image degradation technique causes thesubimage to be converted into an image that has lines that are only onepixel in thickness, while still preserving the continuity in the edges(e.g., lines that connect two vertices) of the original subimage (i.e.,all the pixels of the refined subimage include at least one other pixelin its surrounding eight pixels). FIG. 4 shows an example of a subimagebefore and after it has been refined. In the example, subimage 402 hasnot been refined and subimage 404 has been refined; subimage 404 isintended to depict a numerical character with lines that are only onepixel in thickness but has the continuity of the edges of the original(not refined) subimage preserved.

Returning to FIG. 3, in some embodiments, after the subimage has beenrefined, topological information is obtained from the refined subimage.In various embodiments, topological information refers to informationregarding the vertices of the depiction of the numerical character inthe subimage and the edges between the vertices. A vertex, as usedherein and as used in graph theory, refers to the fundamental unit outof which graphs are formed. An edge, as used herein and as used in graphtheory, refers to a line segment joining two vertices. In variousembodiments, the degree of a vertex refers to the number of edges thatextend from the vertex (as used in the graph theory sense). Suchtopological information is extracted based on the assumption thatinformation regarding vertices and edges thereof is a sufficient basisof data upon which a numerical character can be recognized, despite anywarping or deformation to which the depiction of the numerical characterhas been subject.

In various embodiments, the extracted topological information of thesubimage is matched against preset sets of topological information thatare stored in a database. The preset sets of topological information areassociated with numerical characters from “0” to “9.” In someembodiments, a numerical character is associated with one or more presetsets of stored topological information. If a match is determined betweena set of topological information gathered from a subimage and a set ofstored topological information associated with one of numericalcharacters “0” to “9,” then the depiction of the numerical character inthe subimage is determined to be recognized as the numerical characterassociated with the stored set of topological information. The sets oftopological information can be stored in various forms. For example, therelationships between numerical characters and their correspondingpreset sets of topological information can be stored in a matrix, table,and/or any other data structure. In some embodiments, an individualnumerical character is associated with one preset set of storedtopological information. In some embodiments, the topologicalinformation is extracted and/or stored in the same format as the formatin which the preset topological information is stored so that theextracted information can be matched against the preset topologicalinformation to find a match, if one exists. For example, if the presettopological information were stored in a matrix-form, then the extractedtopological information is also formatted into a matrix-form so that thetwo sets of information can be compared.

In some embodiments, when an extracted set of topological informationcannot be matched against any stored preset set of topologicalinformation, then the extracted set of topological information isstored. In some embodiments, these sets of topological information forwhich a match cannot be found are stored and reviewed later (e.g.,manually by a user to determine whether they are useful). For example,when the stored topological information for which a match was notdetermined is reviewed and it is determined that the set actuallycorresponds to a numerical character, then this relationship can bestored in the database.

FIG. 5 is a flow diagram showing an embodiment of a process of numericalcharacter recognition. In some embodiments, process 500 can beimplemented at system 200.

At 502, a subimage is extracted from a received image, wherein theextracted subimage is associated with one numerical character. 502 canbe described in a manner similar to 302.

At 504, the subimage is refined. In some embodiments, if there aremultiple subimages extracted from the received image, then one subimageis randomly selected to be recognized before the others. Afterrecognition is performed on the first selected subimage, then a secondsubimage can be selected and recognized, and then a third subimage, andso forth, until recognition has been performed on all of the extractedsubimages. In some embodiments, if there are multiple subimages, thenrecognition can be performed on the subimages in parallel. In someembodiments, prior to refining the subimage, preprocessing (e.g., noiseremoval, conversion to monochrome or grayscale) is performed on thesubimage.

In various embodiments, the subimage is refined using mathematicalmorphology techniques. As a result of refining the subimage, thesubimage includes a depiction of a numerical character that is only onepixel in thickness but preserves the continuity of edges from thesubimage prior to refinement.

At 506, one or more vertices and one or more edges associated with thesubimage are extracted.

The refined subimage is one pixel in thickness and all the pixelsinclude at least one other pixel in its eight surrounding pixels. So, itis possible to traverse the pixels of the subimage to locate each of thevertices of the numerical character depicted in the subimage and also toobtain, based on the manner in which two vertices are related, thedegrees of individual vertices. As mentioned before, the degree of avertex corresponds to the number of edges that extend from the vertex.In some embodiments, the number of vertices and related vertex dataassociated with a subimage is referred to as the topological informationassociated with that subimage. FIG. 6A shows an example of the verticesof a subimage with the depiction of numerical character “4.” As shown inthe example, numerical character “4” has 5 vertices A, B, C, D, and E.The degree of vertex A is 2, the degree of vertex B is 2, the degree ofvertex C is 4, the degree of vertex D is 1, and the degree of vertex Eis 1. The degrees of the individual vertices of numerical character “4”of FIG. 6 are represented in Table 1.

TABLE 1 A B C D E 2 2 4 1 1

In some embodiments, the depiction in the refined subimage is traversedin preset directions, such as from left-to-right and from top-to-bottomto detect vertices and the edges between the vertices. For example, thetraversal process can begin from the top-most left filled in pixel ofthe subimage. Usually, the pixel that is traversed first is a vertex.Then the process continues to the next filled in pixel in a left toright and top to bottom fashion. In some embodiments, during a traversalprocess, a vertex can be extracted based on a known technique. Forexample, a vertex can be determined by analyzing the surrounding eightpixels around each filled in pixel of the refined subimage. FIG. 6Bshows three examples of a center filled in pixel of the subimage and itsrespective eight surrounding pixels. 610 shows a 3×3 pixel area wherethe center pixel, pixel 612 is a filled in pixel of the refinedsubimage. In other words, 610 shows center pixel 612 and its eightsurrounding pixels. 610 shows that center pixel 612 is adjacent to onefilled pixel, which indicates that pixel 612 is a vertex (i.e., becausethere is one filled pixel in one of the eight surrounding pixels). 620shows center pixel 614 that is a filled in pixel of the refined subimageand where center pixel 614 has three filled in pixels in its eightsurrounding pixels. Due to the shown arrangement of the threeneighboring filled in pixels (i.e., that are in three directions ofdirectly north, east, south, and west of the center pixel), center pixel614 is determined to be a vertex. 630 shows that center pixel 616 is afilled in pixel of the refined subimage and where center pixel 616 hastwo filled in pixels in its eight surrounding pixels. Due to the shownarrangement of the two neighboring filled in pixels (i.e., that are intwo directions of directly north, east, south, and west of the centerpixel), center pixel 616 is determined to be a vertex. However,techniques of detecting vertices are not limited to only those describedherein.

In some embodiments, the degree associated with a vertex is detectedafter all the vertices are detected. In some embodiments, the degreeassociated with a vertex is determined just after the vertex isdetected. In some embodiments, regardless of how many vertices that adepiction in a refined subimage includes, the technique of detectingvertices and the edges thereof is the same.

Returning to FIG. 5, at 508, the one or more vertices and the one ormore edges are stored.

The vertex and the related edge information detected from a subimage arestored (e.g., at a temporary storage). The vertex and the related edgeinformation can be stored in various forms. Information regardingvertices and each of its respective degrees (e.g., Table 1) can bestored. Also, information regarding whether an edge exists between anytwo vertices can be stored, as shown in the example of Table 2 andexplained as follows: whether an edge (e.g., a connection) existsbetween any two vertices of a subimage can be stored in a matrix/tablethat includes each vertex in a column and in each row; if an edge existsbetween two vertices (e.g., A and B), it is indicated by a “1” at theentry at column A and row B (and also column B and row A) and “0”otherwise.

Referring again to FIG. 6, Table 2 below indicates the vertex andrelated edge information of the numerical character “4.” In Table 2, thevertices (A, B, C, D, and E) of the example in FIG. 6 are in both thecolumns and rows. If a connection/edge exists between two vertices thenthere is a “1” at an entry associated with those two vertices. If aconnection/edge does not exist between two vertices (or if thosevertices are the same vertex), then there is a “0” at an entryassociated with those two vertices. Specifically, for example, becausethere is connection/edge between vertices A and B (as can be seen inFIG. 6), there is a “1” at the intersections of column A with row B andalso column B with row A. Also, because there is no connection/edgebetween vertices A and D, there is a “0” at the intersections of columnA with row D and also column D with row A.

TABLE 2 A B C D E A 0 1 1 0 0 B 1 0 1 0 0 C 1 1 0 1 1 D 0 0 1 0 0 E 0 01 0 0

At 510, it is determined if a closed ring is associated with at least aportion of the one or more vertices and at least a portion of the one ormore edges. In the event that a closed ring is associated with thesubimage, then the position information associated with a central pointof the detected ring is stored. In various embodiments, a closed ringrefers to when three or more vertices and the edges between thosevertices completely enclose an area of the subimage. In someembodiments, the closed ring can be detected using a known technique fordetermining the presence of a closed ring in a graph of vertices andedges. In one example technique, a closed ring detection technique canstart detection at one filled in pixel of the refined image and proceedto a second filled in pixel that is adjacent to the first pixel, andthen to a third filled in pixel that is adjacent to the second pixel andso forth. If the detection sequence of filled in pixels returns back tothe first filled in pixel, then a closed ring has been determined (i.e.,the closed ring is formed by the detected sequence of filled in pixels).In some embodiments, it is kept track of where a detected closed ring isin relation to the other filled in pixels of the refined subimage (e.g.,by assigning a sequence number to each filled in pixel of the refinedsubimage as it is traversed). For example, assuming that the filled inpixels of the subimage is always traversed from left to right and top tobottom, the sequence number of pixel of the closed ring that is firsttraversed can also be stored. FIG. 7 shows an example image of anumerical character “4” with a central point at the center of a closedring. In the example, vertices A, B, and C and the edges associated withat least two of the three vertices form a closed ring (e.g., completelyenclose an area) and point F is a determined point that is central inthe enclosed area. In some embodiments, the central point position isnot necessarily within a geometric center of all the vertices or edgesassociated with the closed ring but is rather some location within theenclosed area. In one example technique, the central point F can bedetermined by first taking the (x, y) coordinates of the pixels thatwere detected to form the closed ring and then finding the average xvalue and the average y value to form the coordinate (i.e., (average x,average y)) of the central point in the closed ring. In another exampletechnique, the central point F can be determined by first determiningthe maximum x value, minimum x value, maximum y value, and minimum xvalue of the pixels that were detected to form the closed ring. Thesefour values (maximum x value, minimum x value, maximum y value, andminimum x value) can be used to form a rectangle. The x- andy-coordinate of the center of this rectangle can be taken as the centralpoint of the closed ring (unless this point is verified to not actuallybe within the enclosed area of the closed ring).

In some embodiments, the central point is stored with the vertex andedge information associated with a subimage (e.g., Table 2). Forexample, if the central point location of the numerical character of “4”were stored with the vertex and edge information for (e.g., Table 2),then the central point location could be included in an additional entryof the table or at a separate location (e.g., as represented by acoordinate). In some embodiments, if a closed ring is not detected fromthe subimage, then data associated with “none” would be stored for theentry associated with central point information.

Regardless of how a closed ring and/or a central point of a closed ringare detected, the detection of one or more closed rings and/or thedetection of one or more central points are useful in determiningwhether a set of topological information extracted from a subimagematches a preset set of stored topological information. For example, thenumerical character “8” is the only numerical character from “0” to “9”that includes two closed rings and also two central points. So, a set oftopological information extracted from a subimage of “8” should match apreset set of stored topological information corresponding to “8,” whichshould be the only preset sets of stored topological information thatinclude information regarding two closed rings and/or two centralpoints. Also, both numerical characters “6” and “9” each include oneclosed ring and one central point. But, assuming that the filled inpixels of the refined subimage is always from left to right and top tobottom, the set of topological information extracted from the subimagewill reflect whether the closed ring was detected earlier in thetraversal process (e.g., for a subimage depicting the numericalcharacter of “9”) or relatively later in the traversal process (e.g.,for a subimage depicting the numerical character of “6”) to distinguishbetween matching the preset set of stored topological informationcorresponding with “6” or “9.”

In some embodiments, 510 is optional and may be omitted.

At 512, it is determined whether the one or more vertices and the one ormore edges associated with the subimage can be matched against a presetset of stored topological information; in the event there is a match, itis determined that the subimage is recognized as a numerical charactercorresponding to the matching preset set of stored topologicalinformation.

In various embodiments, relationships between preset sets of topologicalinformation and their corresponding numerical characters (i.e., “0,”“1,” “2,” “3,” “4,” “5,” “6,” “7,” “8,” or “9”) are stored, for example,in a database. In some embodiments, a numerical character can beassociated with one or more preset sets of stored topologicalinformation. In various embodiments, a stored set of topologicalinformation corresponding to a numerical character includes at leastvertex information (including the degree associated with each vertex),edge information, the relationships (e.g., whether an edge existsbetween two vertices) between the vertices of the numerical character,and, in some embodiments, also information regarding one or more centralpoints. In some embodiments, each set of topological information isstored in one or more matrices or tables, such as Tables 1 and 2, asmentioned above.

In some embodiments, the vertex and related edge information extractedfrom the subimage are converted to the same format as a preset set oftopological information that is stored in the database (if it is notalready in the same format). In some embodiments, the vertex and relatededge information is converted to (if it is not already in the form) ofTables 1 and 2, as explained above. As explained above, Table 1 includesinformation regarding the vertices of the subimage and each of itsrespective degrees and Table 2 includes information regarding whether anedge exists between any two vertices of the depiction in the subimage.

In some embodiments, a portion of the vertex and related edgeinformation extracted from the subimage can be compared to acorresponding portion of a preset set of topological information storedin the database at a time, and if there is a match, then another portionof the extracted information is compared to another correspondingportion of the preset set of topological information and so forth, untilall portions have been compared. If all portions of the extractedinformation match all corresponding portions of the preset set of storedinformation, then the depiction of the subimage is determined to berecognized as the numerical character associated with that matchingpreset set of stored topological information. If there is no matchbetween a portion of the extracted information and a correspondingportion of the preset set of information, then the extracted informationis compared against another preset set of topological information storedin the database.

For example, information regarding the vertices and each of itsrespective degrees of the data extracted from the subimage can becompared to the corresponding information of a preset set of topologicalinformation stored in the database. Specifically, in some embodiments,the order of the extracted vertices (e.g., the order in which thevertices were detected during the traversal process) are also comparedto the order the vertices of the set of stored information to ensure theaccurate order of the vertices of the subimage. If there is a match,then the next comparison can be made between information regardingwhether an edge exists between any two vertices of the data extractedfrom the subimage and the corresponding information of the preset set oftopological information. If there is match, then the third comparisoncan be made between central point(s) extracted from the subimage andcorresponding information of the preset set of topological informationstored in the database. If there is a match, then it is determined thatthe depiction of the subimage is recognized as the numerical characterassociated with the preset set of topological information for which thematches were found.

At 514, it is determined if there are more extracted subimages on whichrecognition is to be performed. In the event that there are subimagesthat were extracted from the received image for which recognition hasnot have been performed, control passes to 502 to process the nextunprocessed subimage. However, if recognition has been performed for allthe subimages, then control passes to 516. Although, in someembodiments, instead of processing each subimage in a series (e.g., oneafter another), each subimage can be processed in parallel.

At 516, a recognized numerical character of the subimage is displayed.In some embodiments, when there are multiple subimages on which therecognition process has been successfully applied, the recognizednumerical characters of the subimages are displayed (e.g., at a screenfor a user that is testing out the recognition process) in the left toright (and top to bottom, if there were multiple rows of numericalcharacters that were recognized) order in which the subimages wereextracted. For example, if the received image was an image thatcontained a phone number (which is read from left to right), then itwould be helpful to display the recognized numerical characters of thephone number in the same left to right order in which each subimage ofthe phone number image was extracted.

FIG. 8 shows an example of one form in which preset sets of topologicalinformation and their corresponding numerical characters can be storedin the database. As shown in the example, column 802 includes numericalcharacters (“0,” “1,” “2,” “3,” “4,” “5,” “6,” “7,” “8,” and “9”) andcolumn 804 includes corresponding visual representations of one or morepreset sets of topological information. In the example, numericalcharacter “0” corresponds to one preset set of topological information,806. While 806 is shown as a visual representation (similar todepictions of refined subimages), the same topological information of806 (i.e., zero vertices, zero edges, one central point) can also bestored in matrices/tables in the database, in some embodiments. In theexample, numerical character “1” corresponds to three preset sets oftopological information 808, 810, and 812. Also, while 808, 810, and 812are shown as visual representations, the same topological information of808 (i.e., two vertices, one edge, zero central points), 810 (i.e.,three vertices, two edges, zero central points), and 812 (i.e., fivevertices, four edges, and zero central points) can also be stored inmatrices/tables in the database, in some embodiments.

FIG. 9 shows a diagram of an embodiment of a system for processingnumerical character recognition. In some embodiments, process 300 can beimplemented at system 900. In some embodiments, system 900 isimplemented in association with a web crawler (e.g., so that the webcrawler could better recognize numerical characters as it crawls throughimages that include depictions of numbers on various embodiments). Insome embodiments, system 900 is a component of a web crawler.

The modules and submodules can be implemented as software componentsexecuting on one or more processors, as hardware such as programmablelogic devices and/or Application Specific Integrated Circuits designedto perform certain functions or a combination thereof. In someembodiments, the modules and submodules can be embodied by a form ofsoftware products which can be stored in a nonvolatile storage medium(such as optical disk, flash storage device, mobile hard disk, etc.),including a number of instructions for making a computer device (such aspersonal computers, servers, network equipments, etc.) implement themethods described in the embodiments of the present invention. Themodules and submodules may be implemented on a single device ordistributed across multiple devices.

Subimage extraction module 902 is configured to extract a subimage froma received image. Each subimage is associated with one numericalcharacter (or in other words, a depiction of a numerical character) from“0” to “9.” In some embodiments, more than one subimage can beextracted.

Image recognition module 904 is configured to perform numericalcharacter recognition processing on each of the one or more extractedsubimages based at least on topological information of each subimage.Image recognition module 904 includes:

Refining submodule 906, which is configured to refine the one or moresubimages. For example, refining submodule 906 can refine a subimageusing mathematical morphology techniques to convert the depiction in thesubimage to be one pixel in thickness.

Topological information extraction submodule 908, which is configured toextract topological information from the one or more subimages.

Matching submodule 910, which is configured to match the extractedtopological information of each subimage against the preset sets oftopological information associated with numerical characters stored in adatabase. If a match is found between the extracted topologicalinformation from a subimage and a preset set of stored topologicalinformation, then it is determined that the depiction in the subimage isrecognized to be the numerical character that is associated with thepreset set of stored topological information.

In some embodiments, system 900 includes saving module 912, which isconfigured to store the extracted topological information if it cannotbe matched against any preset set of topological information that isstored in the database.

FIG. 10 is a diagram showing an embodiment of a system for processingnumerical character recognition. In some embodiments, process 500 can beimplemented at system 1000. In some embodiments, system 1000 isimplemented in association with a web crawler. In some embodiments,system 1000 is a component of a web crawler.

System 1000 includes certain submodules (subimage extraction submodule902, refining submodule 906, topological information extractionsubmodule 908 and matching submodule 910) that were also present insystem 900 and will not be described again.

Degree extraction submodule 914 is configured to extract the one or morevertices and also the one or more edges of a refined subimage.

Connection relation recording submodule 916 is configured to store(e.g., in a temporary storage) the extracted vertex and related edgeinformation including information regarding the connections between thevertices of the subimage (e.g., whether there is an edge present betweentwo vertices).

Position information recording submodule 918 is configured to determinewhether there is a closed ring in a subimage and if so, to record aposition of a central point of a detected closed ring.

Demonstration module 920 is configured to present the numericalcharacters recognized from the one or more subimages at a display. Insome embodiments, the recognized numerical characters are displayed fromleft to right and top to bottom, in the same order in which thecorresponding subimages were extracted.

It must be explained that the individual embodiments in thisSpecification have been described using an incremental method, andwhatever is emphasized in the explanation of each embodiment are thedifferences from the other embodiments, and the shared and similar partsof individual embodiments can be included by reference. Certaindescriptions of embodiments of system can be supplemented with thedescriptions of the method embodiments.

Finally, it must be explained that the terms “include,” “contain” or anyother variants thereof are aimed at covering non-exclusive inclusion,hence, such that, a process, method, system or device that includes anumber of essential modules, not only includes those modules but alsoincludes other essential modules that were not explicitly listed; orincludes essential modules that are inherent to this process, method,system, or device. In the absence of other restrictions, essentialmodules qualified by the phrase “including one . . . ” do not precludethe existence of other identical essential modules within theabove-mentioned process, method, system, or device.

Above, we have provided a description of the method of digital numericalcharacter recognition, system, and crawler server offered by thispresent disclosure; specific examples were used in the text in order toelaborate on the principle and realization method of this presentdisclosure. The explanation of the above embodiments is only forassistance in understanding the method of this present disclosure andits core concepts. At the same time, according to the concept of thispresent disclosure, regular technical experts in the field may makemodifications to specific modes of realization and utilization scope; tosummarize the above, this specification contents must not be understoodto limit this present disclosure.

Although the foregoing embodiments have been described in some detailfor purposes of clarity of understanding, the invention is not limitedto the details provided. There are many alternative ways of implementingthe invention. The disclosed embodiments are illustrative and notrestrictive.

What is claimed is:
 1. A system, comprising: a processor configured to:extract a subimage from a received image comprising informationpertaining to a plurality of numerical characters, wherein the extractedsubimage is associated with one of the plurality of numericalcharacters; perform recognition based at least in part on a set oftopological information associated with the subimage, including: processthe subimage to obtain the set of topological information associatedwith the subimage, wherein the processing of the subimage includes:obtain the set of topological information by extracting one or morevertices and one or more edges associated with the subimage; and add acentral point representing a closed ring to the set of topologicalinformation; compare the set of topological information associated withthe subimage with a preset set of stored topological information,comprising: compare an order in which the extracted one or more verticeswere detected during a traversal process with an order of vertices ofthe preset set of stored topological information, the traversal processbeginning from a predefined pixel of the subimage and continuing to anext filled in pixel in a predefined order; determine that in the eventthat the set of topological information associated with the subimagematches the preset set of stored topological information, the subimageis associated with a recognized numerical character associated with thepreset set of stored topological information; and a memory coupled tothe processor and configured to provide the processor with instructions.2. The system of claim 1, wherein the processor is further configured torefine the subimage.
 3. The system of claim 2, wherein to refine thesubimage includes to apply image degradation to the subimage.
 4. Thesystem of claim 1, wherein the preset set of stored topologicalinformation is stored in a database.
 5. The system of claim 1, whereinthe processor is further configured to display the recognized numericalcharacter.
 6. The system of claim 1, wherein the recognized numericalcharacter is one of the following: 0, 1, 2, 3, 4, 5, 6, 7, 8, or
 9. 7. Amethod, comprising: extracting a subimage from a received imagecomprising information pertaining to a plurality of numericalcharacters, wherein the extracted subimage is associated with one of theplurality of numerical characters; and performing recognition based atleast in part on a set of topological information associated with thesubimage, including: processing the subimage to obtain the set oftopological information associated with the subimage, wherein theprocessing of the subimage includes: obtaining the set of topologicalinformation by extracting one or more vertices and one or more edgesassociated with the subimage; and adding a central point representing aclosed ring to the set of topological information; comparing the set oftopological information associated with the subimage with a preset setof stored topological information, comprising: comparing an order inwhich the extracted one or more vertices were detected during atraversal process with an order of vertices of the preset set of storedtopological information, the traversal process beginning from apredefined pixel of the subimage and continuing to a next filled inpixel in a predefined order; determining that in the event that the setof topological information associated with the subimage matches thepreset set of stored topological information, the subimage is associatedwith a recognized numerical character associated with the preset set ofstored topological information.
 8. The method of claim 7, furthercomprising refining the subimage.
 9. The method of claim 8, whereinrefining the subimage includes applying image degradation to thesubimage.
 10. The method of claim 7, wherein the preset set of storedtopological information is stored in a database.
 11. The method of claim7, further comprising displaying the recognized numerical character. 12.The method of claim 7, wherein the recognized numerical character is oneof the following: 0, 1, 2, 3, 4, 5, 6, 7, 8, or
 9. 13. A computerprogram product, the computer program product being embodied in atangible non-transitory computer readable storage medium and comprisingcomputer instructions for: extracting a subimage from a received imagecomprising information pertaining to a plurality of numericalcharacters, wherein the extracted subimage is associated with one of theplurality of numerical characters; and performing recognition based atleast in part on a set of topological information associated with thesubimage, including: processing the subimage to obtain the set oftopological information associated with the subimage, wherein theprocessing of the subimage includes: obtaining the set of topologicalinformation by extracting one or more vertices and one or more edgesassociated with the subimage; and adding a central point representing aclosed ring to the set of topological information; comparing the set oftopological information associated with the subimage with a preset setof stored topological information, comprising: comparing an order inwhich the extracted one or more vertices were detected during atraversal process with an order of vertices of the preset set of storedtopological information, the traversal process beginning from apredefined pixel of the subimage and continuing to a next filled inpixel in a predefined order; and determining that in the event that theset of topological information associated with the subimage matches thepreset set of stored topological information, the subimage is associatedwith a recognized numerical character associated with the preset set ofstored topological information.
 14. The computer program product ofclaim 13, further comprising refining the subimage.
 15. The computerprogram product of claim 14, wherein refining the subimage includesapplying image degradation to the subimage.
 16. The computer programproduct of claim 13, wherein the preset set of stored topologicalinformation is stored in a database.
 17. The computer program product ofclaim 13, further comprising displaying the recognized numericalcharacter.
 18. The computer program product of claim 13, wherein therecognized numerical character is one of the following: 0, 1, 2, 3, 4,5, 6, 7, 8, or
 9. 19. The system of claim 1, wherein the comparing ofthe set of topological information associated with the subimage with thepreset set of stored topological information further comprises: comparean edge existing between two vertices with the preset set of storedtopological information; and compare the central point with the presetset of stored topological information.